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Abstract. We construct uncertainty intervals for weak Poisson signals in the 
presence of background. We consider the case where a primary experiment yields a 
realization of the signal plus background, and a second experiment yields a realization 
of the background. The data acquisitions times, for the background-only experiment, 
Tb g , and the primary experiment, T, are selected so that their ratio, -If, varies 
from 1 to 25. The upper choice of 25 is motivated by an experimental study at 
the National Institute of Standards and Technology (NIST). The expected number 
of background counts in the primary experiment varies from 0.2 to 2. We construct 
90 and 95 percent confidence intervals based on a propagation-of-errors method 
as well as two implementations of a Neyman procedure where acceptance regions 
are constructed based on a likelihood-ratio criterion that automatically determines 
whether the resulting confidence interval is one-sided or two-sided. In one of the 
implementations of the Neyman procedure due to Feldman and Cousins, uncertainty 
in the expected background contribution is neglected. In the other implementation, 
we account for random uncertainty in the estimated expected background with a 
parametric bootstrap implementation of a method due to Conrad. We also construct 
minimum length Bayesian credibility intervals. For each method, we test for the 
presence of a signal based on the value of the lower endpoint of the uncertainty 
interval. In general, the propagation-of-errors method performs the worst compared to 
the other methods according to frequentist coverage and detection probability criteria, 
and sometimes produces nonsensical intervals where both endpoints are negative. The 
Neyman procedures generally yield intervals with better frequentist coverage properties 
compared to the Bayesian method except for some cases where = 1. In general, 
the Bayesian method yields intervals with lower detection probabilities compared to 
Neyman procedures. One of main conclusions is that when -^r- is 5 or more and 
the expected background is 2 or less, the FC method outperforms the other methods 
considered. For = 1,2 we observe that the Neyman procedure methods yield false 
detection probabilities for the case of no signal that are higher than expected given 
the nominal frequentist coverage of the interval. In contrast, for = 1,2, the false 
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detection probability of the Bayesian method is less than expected according to the 
nominal frequentist coverage. 

PACS numbers: 02.50.-r,07.90.+c,07.05.Kf,07.81.+a,14.60.Lm,29.85.- 
c,29.85.Fy,95.35.+d 
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1. Introduction 

We consider experiments where instruments yield count data that can be modeled as 
realizations of a Poisson process with expected value jj, s + jj, B where us is the expected 
contribution due to a signal of interest, and /ie is the expected contribution of a 
background process. That is, 

n bs ~ Poi{ns + //b)- (1) 

Given the measured value n b. s and an estimate of \xb from an independent background- 
only experiment, we construct uncertainty intervals (confidence intervals and Bayesian 
credibility intervals) for fx S - The statistical problem we study occurs in a variety 
of application areas including: particle and astroparticle physics [1-9], isotopic ratio 
analysis (when the major isotope is large enough so that most of the variability in the 
ratio is due to the minor isotope) [10,11], detection of low-level radiation [12-15], and 
aerosol science and technology [16]. 

Here, we focus on the case where the signal is weak and consider the case where 
the ratio of the data acquisition time for the background-only measurement Tb g and the 
data acquisition time for the primary experiment T varies from 1 to 25. This upper 
value of ^ = 25 was motivated by an experimental study at the National Institute of 
Standards and Technology (NIST), as were the values of \i s and /i B that we consider. 
For such cases, we demonstrate that the standard propagation-of-errors (POE) method 
yields confidence intervals with poor coverage properties. Sometimes the POE method 
produces intervals where the upper and/or lower endpoints are negative. As an aside, 
for the special case where — 1, one can construct a confidence interval for /is based 
on a Bessel function approach that has better coverage properties than does the POE 
method [16]. However, for the general case where ^ 1, this method is not applicable. 
Hence, we do not include the method described in [16] in our study. 

In addition to the POE method, we study the relative performance of three 
other methods for constructing uncertainty intervals. The first method [17] is an 
implementation of a frequentist Neyman procedure [18] developed by Feldman and 
Cousins. In this method, which we refer to as the FC method, /ib is assumed to be 
known. For each of many discrete values of fis, acceptance regions are constructed based 
on a likelihood-ratio criterion. Given the intersection of the actual measured value with 
these regions, one constructs a confidence interval for fis- In our studies, we estimate 
Hb from background-only experiments. In [19], the FC method was extended to account 
for systematic uncertainties in We denote this method as the randomized Feldman 
Cousins (RFC) method because hb is treated as a random nuisance parameter. In 
this work, we implement a version of the RFC method where uncertainty in /i^ is due 
to Poisson counting statistics variation in a background-only experiment that gives a 
direct measurement of hb- In the RFC method, we simulate realizations of the nuisance 
parameter hb with a parametric bootstrap method [20]. In both the FC and the RFC 
methods, the upper and lower endpoints are determined automatically. 
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We also determine the posterior probability density function (posterior pdf ) for fi s 
with a Bayesian method [21,22] following Loredo's treatment of the same problem in 
[23]. Loredo did not discuss how to select the endpoints of the credibility interval. Here, 
given that the integrated posterior pdf has a particular value (equal to the nominal 
frequentist coverage probability), we determine the endpoints by minimizing the length 
of the credibility interval. As an aside for the special case where /is is known, Roe and 
Woodroofe [24] determined minimum length Bayesian credibility intervals assuming a 
uniform prior for /is and studied the frequentist coverage properties of their intervals. 

Bayesian credibility intervals and frequentist confidence intervals are conceptually 
different. To illustrate, consider a one-dimensional parameter estimation problem. 
Frequentist confidence intervals are constructed so that, ideally, the true value of 
the parameter falls within the confidence interval determined from any independent 
realization of data with some desired coverage probability. In contrast, Bayesian 
credibility intervals are constructed by modeling the parameter of interest as a random 
variable. Given a prior probability model for the parameter of interest and a likelihood 
model for the data given the parameter, Bayes theorem yields the conditional probability 
density function of the parameter given the observed data. Based on this conditional 
pdf (called the posterior pdf), one constructs credibility intervals. By design, Bayesian 
credibility intervals are not constructed with frequentist coverage in mind. Although the 
conceptual foundations of frequentist and Bayesian inference are different, frequentist 
coverage is widely accepted as an empirical measure of the performance of not only 
frequentist confidence intervals but Bayesian credibility intervals as well [22,25,26]. In a 
highly regarded textbook on Bayesian data analysis, Gelman, Carlin, Stern and Rubin 
remark (page 111 of [22]) 

Just as the Bayesian paradigm can be seen to justify simple 'classical' 
techniques, the methods of frequentist statistics provide a useful approach 
for evaluating the properties of Bayesian inferences- their operating 
characteristics- when these are regarded as embedded in a sequence of repeated 
samples. 

In this frequentist coverage study, we simulate realizations of data given /is and /ib, 
and quantify the probability that us falls in the interval determined from the simulated 
data. In frequentist statistics, the relationship between a confidence interval and a 
hypothesis test is well known. We exploit this relationship and test the null hypothesis 
that fis = against the alternative hypothesis fis > 0, based on the value of the lower 
endpoint of the uncertainty interval. We reject the null hypothesis if the lower endpoint 
is greater than 0. Thus, the probability that the lower endpoint of an interval is greater 
than is a signal detection probability. As a caveat, we do not claim that this procedure 
is the most powerful test of our hypothesis. 

In Section 2, we define our measurement model and describe how we determine 
uncertainty intervals using each of the four methods. In this study, the background 
parameter \i B ranges from 0.2 to 2 and the signal parameter fi s ranges from to 
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20. In Section 3, we study the coverage properties of uncertainty intervals for a 
variety of cases. We also determine detection probabilities for a signal of interest. In 
general, the propagation-of-errors method performs the worst compared to the other 
methods according to frequentist coverage and detection probability criteria. Further, 
the propagation-of-errors method sometimes produces nonsensical intervals where both 
endpoints are negative. The Neyman procedures generally yield intervals with better 
frequentist coverage properties compared to the Bayesian method except for the case 

rp 

where -|r = 1 and there are 1 or more expected background counts in the primary 
experiment. In general, the Bayesian method yields intervals with lower detection 

rp 

probabilities compared to Neyman procedures. When is 5 or more, the FC method 
yields intervals with the highest detection probabilities and best coverage properties 
in general. However, for -|r = 1, 2 both the Neyman procedure methods yield false 
detection probabilities for the case of no signal that are higher than expected given 

rp 

the nominal frequentist coverage of the interval. In contrast, for — 1,2, the false 
detection probability of the Bayesian method is less than expected according to the 
nominal frequentist coverage. 



2. Measurement Model and Uncertainty Intervals 

In our simulation study, we consider an experiment where a realization of the signal 
of interest plus background, n b. s is observed during a time interval T. In a separate 

rp 

experiment of duration Tt, g , where -|r varies from 1 to 25, we measure a realization of 
background n^g. We denote the data as d = (n b s , n^). The expected values of n b s and 

rp 

Ub g are + A*b and /is^f, respectively, where /is is the expected contribution from 
the signal of interest, and hb is the expected contribution from the background. We 
model measurements of n f, s and n^g as independent Poisson random variables. Hence, 
the likelihood function of the data is P(d\fj,s, I^b), where 

P(d\» s , »b) = fa + VeY*- ^B^±M x (2) 
(n T bg exp[-(^)] 

2.1. Feldman Cousins Method 

In the FC method, one determines confidence intervals with a Neyman procedure 
assuming exact knowledge of /ib- In our study, we set //# to an empirical estimate 
fiB, where 

T 

Ab = 7f^n bg . (3) 

J-bg 

Hence, the variance of fis is 

VAR(ji B ) = (£-)vl b , (4) 

J-bg 
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and the standard deviation of //^ is 



Thus, the fractional uncertainty of the estimate of //£ is 




(5) 




(6) 



In Figure 1, we plot probability density functions for the estimated background 
when = 25 for fi B = 0.2, 1, and 2. 

The FC method [17] produces a confidence interval for us under the assumption 
that the assumed background (fi B in our case) equals the true background fi B . For 
various values of fj,s, we construct an acceptance region in n space. For each integer 
value of n, we compute the conditional probability P(n\fi s , fi B ) and P(n\fis, fibest), where 
ftbest = max(0, n — fi B ) and 



We include values of n in the acceptance region with the largest values of R. For 
construction of a 100 x p % confidence interval, we add values until the sum of the 
P(n\fj,s, fi B ) terms is p or greater. The lower and upper endpoints of the confidence 
interval for us are the minimum and maximum values of p,$ that yield acceptance 
regions that include the observed value n b s . For fixed n b s , due to the discreteness of 
n, the upper endpoint of the interval is not always a decreasing function of /i B . In 
[17], Feldman and Cousins lengthened their intervals so that the upper interval was a 
non- decreasing function of /i B . In this work, we do not adjust our intervals. 

2.2. Extension of Feldman Cousins: Uncertain Background 

In the RFC method, the value of \i B is a random nuisance parameter. In our analysis, we 
assume that uncertainty in \i B is due to random variation alone, i.e., counting statistics. 
If there were systematic error, it could be incorporated into the analysis. However, we 
do not do this. 

The procedure to construct a confidence interval is very similar to the FC method. 
For each value of /is, we compute an acceptance region like before, but we replace 
P(n\/is, As) and P(n\fis, fibest) with an estimate of their expected values when one 
accounts for uncertainty in fi B . 

One way to do this would be to simulate realizations of fi B with a parametric 
bootstrap [20] method and determine the mean value of P(n\/is, As) and P(n\fis, frbest) 
from all the realizations. In this approach, the kth. bootstrap replication of n^, nl g {k) 



P(n\fj, s ,fJ-B) = (/Us + Ab) 



n 



exp[-(/ig + /*£>)] 



(7) 



From these, we form the ratio R, where 

P(n\fi s ,p, B ) 
it — — — — : -. 

P(n\fi S ,fibest> 



(8) 
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is simulated by sampling from a Poisson distribution with expected value equal to n bg . 
That is, 

nt g (k)~Poi(n bg ). (9) 
Given nl (k), the kth bootstrap replication of £13, is 

/£(*) = 7^< 9 (k), (10) 

and the kth. bootstrap replication of ji best is ft>test( k ) = ™^(0,n — fi* B (k)). Thus, the 
kth bootstrap replication of P(n\iis, A) is 

P(n\ns,H B (k)) = [Hs + /%(&)) ~ { , (11) 

and the fcth bootstrap replication of P(n\ /is, jibest) is 

«„«(*)) = + M ^^"' . (12) 



From all .fT bootstrap replications, we determine the following mean values 

1 K 

P(n\ns, M = j7^2 P ( n l/ is ' #b(*0) (I 3 ) 



k=i 

and 

1 * 

P(n\ns, jlbest) = — P ( n \VS, iKesti k )) (I 4 ) 
fe=l 

In this Monte Carlo implementation of the RFC method, one replaces P(n\fis, As) and 
P(n\/is, frbest) with the right-hand sides of Eqns. 13 and 14. 

To reduce computer run time, we do not implement a Monte Carlo version of 
the RFC. Instead, we determine the left-hand sides of Eqns. 13 and 14 by numerical 
integration. For instance, we evaluate the left-hand side of Eq. 13 as 

P(n\ns,ii B )= Y p ( n \ns,fj>* B = Tfr-)w{k) (15) 

J-bg 



k=k t , 



where w(k) is 



exp(-n bg )n k b 

w ( k ) = £j • (16) 

To speed up the algorithm, we select k\ ow and k^ so that the sum of the w(k) terms 
agrees with 1 to within approximately 10~ 8 . We use a similar method to determine the 
left-hand side of Eq. 14. 



2.3. Bayesian Method 

Following [23], we determine a Bayesian credibility interval for given measurements 
of n obs and n bg . In this approach, the priors p(^b) and p(hs\i^b) are both uniform from 
to a large positive constant. Results are presented for the limiting case where this 
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positive constant approaches infinity. Based on a Bayes Theorem argument, one can 
show that 

p(Hs^B\n bs,n bg ) oc p(iiB\n bg )p(n obs \iis, Hb), (17) 
where the posterior pdf for given n bg is 

= -t^t — -: , (18) 

and p(n obs \/is, Hb) is the Poisson likelihood function 

p(n obs \fi s ,I^B) = exp(-(// s + fi B ))(fj, s + fi B ) n ° bs /n obs \. (19) 

See Figure 2 for examples of Eq. 18. 

Further, marginalizing with respect to we get 



(/i 5 ) l exp(-/i 5 ) 



p(Vs\n bs,n bg ) = }Ci , (20) 



i=0 



where 



/-i _|_ ^6£\j (ra h s +n;, g -i)! 
^ _ ___ ^_ {n obs -i)\ 

% (1 _l_ ^9 V-j ( n obs+n b g-j)\ ' ^ ' 

^j=0\ L "T T J (n oi ,-j)! 

See [23] for more details of this derivation. 

In this study, we determine the endpoints of 90 % and 95 % credibility intervals as 
follows. For both the 90 % and 95 % cases, we determine the maximum lower endpoint 
of the one-sided interval l max - I n an optimization code, for each trial value of the lower 
endpoint I (where < I < l max ) we determine the upper endpoint u such that the 
integral of the posterior pdf from I to u equals the nominal frequentist coverage. We 
determine the lower endpoint / that minimizes u — I. If the optimal value of I is less 
than the specified numerical tolerance (10~ 6 ) of the optimization algorithm, we set it to 
0. 



2.4- Propagation-of-Errors Method 

We compute two-sided confidence intervals with a standard propagation-of-errors (POE) 
method that has a continuity correction. The 1 — a level POE confidence interval for 
Us is frs ± + 0.5) where 

fis = n obs - fi B , (22) 

and 

T 

°ls = n bs + {—f^bg- (23) 

For levels of 0.90 and 0.95, z a / 2 = 1.64 and 1.96, respectively. We expect that this 
method will yield confidence intervals with coverage close to the desired nominal values 
for the asymptotic case where the signal-to-noise ratio of the data is high. As a caveat, 
continuity corrections are typically introduced when constructing confidence intervals for 
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the case where there is no background [27] rather than the more general case considered 
here. 

In our simulation experiment, the POE method can yield nonsensical results where 
one or both endpoints are negative (Table 1). In our coverage studies, we treat negative 
endpoints as 0. Hence, if both endpoints are negative, the resultant interval is treated 
as (0,0). In physics and astroparticle physics experiments where one hopes to discover 
a new particle, null resuls are common and experimenters provide upper limits. If both 
endpoints are negative, one can not set a reasonable upper limit. Hence the POE method 
is clearly unacceptable for low count data sets. 

3. Simulation Experiments 

In Table 1, we list some realizations of data and associated intervals constructed to have 
nominal coverage of 90 % for the four methods. Based on 2000 realizations of data for 

rp 

each of various choices of -|r and fxs and we determine the frequentist coverage as 
the fraction of the intervals that cover the true value of /is for each method (Tables 
2-7). We also estimate detection probabilities for the different methods for levels 0.90 
and 0.95 (Tables 8-13). 

To start, we consider 90 percent intervals for the case where //# = 1. In Figures 

rp 

3 and 4, we show coverage and detection probabilities as a function of and -!jr 

rp 

for this case. In general, when -tjj- — 1, the coverage properties of the FC and RFC 
methods are poor at low value of /is- In Figures 5 and 6, we show the false detection 
probabilies for all cases for //£ — 0. For = 1,2, both the RFC and FC methods have 
false detection probabilities that are higher than predicted according to the nominal 
frequentist coverage of the intervals. Hence, reporting a discovery based on an analysis 
with the FC or RFC method should be treated with great caution for cases where 

— 1,2. For > 5, the false detection probabilities of the FC and RFC methods 
are generally slightly less than their associated nominal target values. In contrast, for 

rp 

all values of -|r, the false detection probabilities of the Bayesian method are less than 
the values predicted by the nominal frequentist coverage. In Figures 7-10, we display 
coverage and detection probabilities for all cases considered in our simulation study. 

In Tables 14 and 15, we list the root-mean-square (RMS) deviation between the 
observed and nominal frequentist coverage probabilities as a function of fi B . We include 
results for fi s < 10. According to our coverage and detection probability criteria, the 
POE method performed the least well of all methods. This is not a surprise since the 
poor performance of the POE method for low-count situations is well known. In general, 
the Bayesian method yielded intervals with the lowest detection probabilities compared 
to the FC and RFC methods. According to the RMS coverage criterion, the coverage 
properties of the Bayesian intervals are inferior to the intervals produces by the FC and 
RFC methods for most cases. The exception to this pattern was for the case where 

rp 

— 1 and n B = 1,2. The FC and RFC method had better coverage compared to 
Bayesian method for jj, B = 0.2 for all values of ^ considered. 
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For ^ = 1,2 the coverage properties of the RFC method where slightly better 
than those of the FC method for = 1,2. However, for greater than or equal 
to 5, the FC method yields intervals with superior coverage and detection probabilities 
compared to the RFC and Bayesian methods. 

3.1. Comments 

For fixed as fi$ increases, we sometimes observe nonmonotic trends in coverage. In 
other studies such as [24], nonmonotic trends were also observed. 

We expect the FC method to yield poor results when the 1-sigma uncertainty in 

rp 

the estimated background (Eq. 5) is large. For — \ and /i£ = 0.2, 1, 2, Eq. 5 yields 
absolute uncertainties of 0.45, 1 and 1.41, and Eq. 6 yields fractional uncertainties of 224 

rp 

%, 100 % and 71 % respectively. For — 5, the absolute and fractional uncertainties 
are 0.2, 0.45 and 0.63, and 100 %, 45 % and 31 % respectively. For = 25, the 
absolute and fractional uncertainties are 0.09, 0.20 and 0.28, and 45 %, 20 % and 14 % 
respectively. 

It is plausible that the performance of the FC method depends solely on the Eq. 5 
uncertainty of the background estimate. However, comparison of the coverage properties 
of the FC intervals for the case where -|t = 1,/xb = 0.2 and for the case where — 5, 
Hb — 1, suggest a more complicated picture. For the first case, the FC intervals have 
poor coverage at low values of /is (Tables 2,5). For the second case, the intervals have 
good coverage at all /is (Tables 3,6). However, the standard deviation of the estimated 
background is the same for both cases. Perhaps this result is due to differences in the 
shapes of the background estimate pdfs for -^f = 1 and -^f = 5. 

In the POE method, we approximate the distribution of the background-corrected 
estimate of fis, n b. s — £i>b, as a normal (Gaussian) random variable. For the special case 
where fis = 0, a common rule of thumb is that the normal approximation is reasonable 
when the expected value of n b s is greater than about 10 [27]. From this, we conclude 
that if the expected values of of n t> s and n^g both exceed 10, the Gaussian assumption 
seems reasonable. As a caveat, the adequacy of the normal approximation depends on 
the goal of the analysis. For instance, constructing a confidence interval with nominal 
coverage of 0.99 is a more demanding task than constructing an interval with nominal 
coverage of 0.90. For the cases studied here where the nominal coverage is 0.90 or 0.95, 
the POE intervals had coverage close to the desired nominal values when /is was greater 
than about 5. 

In our implementation of the Bayesian method, we specify uniform priors for fis 
and hb and construct a minimum length credibility interval. Roe and Woodroofe [24] 
determined a minimum length credibility interval based on a uniform prior for us for the 
simpler problem where \xb was assumed to be known. Hence, our study can be regarded 
a generalization of [24] to the case where \ib is not known exactly. As a caveat, in 
a Bayesian approach, one could consider other priors. For a given experiment, it is 
possible that other priors might be more appropriate than the uniform prior considered 
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here. How well such alternative Bayesian schemes would perform relative to the one 
studied here is beyond the scope of this study. 

As remarked earlier, we did not adjust our intervals to ensure that the upper 
endpoint of the FC and RFC intervals are nondecreasing functions of \ib- It is possible 
that such an adjustment might improve the performance of the FC and RFC methods. 
Also, the FC method of computing the likelihood-ratio term R may not be the best 
procedure [28-31]. 

4. Summary 

In this work, we studied four methods to construct uncertainty intervals for very 
weak Poisson signals in the presence of background. We considered the case where 
a primary experiment yielded a realization of the signal plus background, and a second 
experiment yielded a realization of the background. The duration of the background- 
only experiment T hg and and the duration of the primary experiment T were selected 

rp rp 

so that varied from 1 to 25. This choice of -|£ =25 was motivated by experimental 
studies at NIST. The values of the expected background /jl b varied from 0.2 to 2. The 
choice of the range was also motivated by NIST experiments. 

We constructed confidence intervals based on the standard propagation-of-errors 
method as well as two implementations of a Neyman procedure due to Feldman and 
Cousins (FC) and Conrad (RFC). In the FC method, uncertainty in the background was 
neglected. In our implementation of the RFC method, uncertainty in the background 
parameter was accounted for. In both of these methods, acceptance regions were 
determined for each value of the expected signal rate based on a likelihood-ratio 
ordering principle. Hence, the upper and lower endpoints of the confidence intervals 
were automatically selected. We also constructed minimum length Bayesian credibility 
intervals. 

According to our coverage and detection probability criteria, the POE method 
performed the least well of all methods. In general, the Bayesian method yielded 
intervals with the lowest detection probabilities compared to the FC and RFC methods 
(Tables 8-13, Figures 4,5,6,8 and 10). According to an RMS criterion, the coverage 
properties of the Bayesian intervals were inferior to the intervals produces by the FC 
and RFC methods (Tables 14 and 15) for most cases. The exception to this pattern was 

rp 

for the case where — 1 and fi B = 1,2. 

The FC and RFC methods had better coverage compared to the Bayesian method 
for \ib = 0.2 for all values of -|r considered. We expect similar results for \xb < 0.2. 
We interpret this result as evidence that when expected number of background counts 
is 0.2 or less, the FC method (which neglects uncertainty in the background) works well 
because uncertainty in the observed background is not significant compared to other 
sources of uncertainty that affect the interval. 

rp 

For — 1,2 the coverage properties of the RFC method where slightly better 
than those of the FC method for fi B = 1,2. However, for greater than or equal to 
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5, the FC method yielded intervals with superior coverage and detection probabilities 
compared to the RFC and Bayesian methods. We attribute the good performance of the 
FC method to the fact that uncertainty in the estimated background is not significant 
compared to other sources of uncertainty that affect the interval when /i# < 2 and 
> 5. The relative performance of the three methods for /i# > 2 is an open question. 
We speculate that for the FC method to yield a result superior to the Bayesian or RFC 

rp 

method for much larger than 2, may have to be larger than 5 in order to reduce 
the uncertainty of estimated background to a sufficiently low level. 

rp 

As a caveat, for -|r = 1,2, both the RFC and FC methods had false detection 
probabilities that were higher than predicted according to the nominal frequentist 

rp 

coverage of the intervals for = 1,2 (Figures 5,6). Hence, reporting a discovery 
based on an analysis with the FC or RFC method should be treated with great caution 
for cases where -|r = 1,2. For > 5, the false detection probabilities of the FC and 
RFC methods were generally slightly less than their associated nominal target values. 

rp 

In contrast, for all values of the false detection probabilities of the Bayesian method 
were less than the value predicted by the nominal frequentist coverage. 

Acknowledgements 

We thank H.K. Liu, L.A. Currie and the anonymous reviewers of this work for useful 
comments. 



Uncertainty Intervals for Weak Poisson Signals 



13 



Table 1. Upper and lower endpoints of uncertainty intervals with nominal frequentist 
coverage of 0.90. The intervals are determined from simulated values of n obs and n bg . 
For informational purposes, we list /ig and jiB- 
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25 


0.1 


2.0 





46 


0.00 


2.30 


0.00 


1.15 


0.00 


1.15 


-2.79 


-0.89 


25 


5.0 


1.0 


2 


16 


0.00 


4.70 


0.00 


5.27 


0.00 


5.27 


-1.48 


4.20 


25 


10.0 


0.2 


9 


7 


4.57 


14.62 


4.08 


15.01 


4.08 


15.04 


3.28 


14.16 
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Table 2. Estimated coverage probabilities of uncertainty intervals with nominal 
frequentist coverage of 0.90. Approximate 68 percent uncertainty due to sampling 
variability given for cases where estimated coverage probability is greater than or 

less than 1. ^ = 1. 



Us Bayesian FC RFC POE 



0.0 





.2 




1 






0. 


,857± 


0.008 


0.857± 





.008 


1 






0.1 





.2 




1 






0. 


.797± 


0.009 


0.798± 


0. 


.009 


1 






0.2 





.2 




1 






0. 


948± 


0.005 


0.948± 





,005 


1 






1.0 





.2 





,998± 





.001 


0. 


908± 


0.006 


0.908± 





.006 


0.751± 


0, 


.010 


2.0 





.2 





.995± 





.002 


0. 


960± 


0.004 


0.961± 





.004 


0.887± 





.007 


5.0 





.2 





.867± 





.008 


0. 


923± 


0.006 


0.933± 





.006 


0.869± 


0, 


.008 


10.0 





.2 





.906± 


0. 


.007 


0. 


904± 


0.007 


0.906± 





.007 


0.907± 





,007 


20.0 





.2 





,911± 


0. 


.006 


0. 


904± 


0.007 


0.908± 





.006 


0.911± 


0, 


.006 


0.0 


1 


.0 





.990± 





.002 


0. 


,728± 


0.010 


0.748± 





.010 


0.991± 





.002 


0.1 


1 


.0 





.992± 


0. 


.002 


0. 


,708± 


0.010 


0.733± 





.010 


0.985± 


0, 


.003 


0.2 


1 


.0 





.986± 





.003 


0. 


861± 


0.008 


0.862± 


0. 


.008 


0.982± 





,003 


1.0 


1 


.0 





.994± 





.002 


0. 


841± 


0.008 


0.856± 





,008 


0.906± 





,007 


2.0 


1 


.0 





.981± 





.003 


0. 


909± 


0.006 


0.927± 





.006 


0.924± 


0, 


.006 


5.0 


1 


.0 





,900± 





.007 


0. 


901± 


0.007 


0.922± 





.006 


0.916± 





.006 


10.0 


1 


.0 





.896± 





.007 


0. 


906± 


0.007 


0.921± 





.006 


0.917± 


0, 


,006 


20.0 


1 


.0 





.903± 


0. 


.007 


0. 


,886± 


0.007 


0.901± 


0. 


.007 


0.903± 





,007 


0.0 


2 


.0 





,964± 





.004 


0. 


,786± 


0.009 


0.835± 





.008 


0.974± 


0, 


.004 


0.1 


2 


.0 





.972± 





.004 


0. 


763± 


0.010 


0.828± 





.008 


0.949± 





.005 


0.2 


2 


.0 





,968± 


0. 


.004 


0. 


,843± 


0.008 


0.857± 





.008 


0.951± 


0, 


.005 


1.0 


2 


.0 





,980± 





.003 


0. 


,838± 


0.008 


0.867± 


0. 


.008 


0.939± 





,005 


2.0 


2 


.0 





.978± 





.003 


0. 


866± 


0.008 


0.930± 





.006 


0.926± 





.006 


5.0 


2 


.0 





,910± 





.006 


0. 


,886± 


0.007 


0.927± 





,006 


0.924± 


0, 


.006 


10.0 


2 


.0 


0. 


.872± 





.007 


0. 


,872± 


0.007 


0.912± 





.006 


0.913± 





.006 


20.0 


2 


.0 





.907± 


0. 


.007 


0. 


,886± 


0.007 


0.908± 





.006 


0.910± 


0, 


,006 
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Table 3. Estimated coverage probabilities of uncertainty intervals with nominal 
frequentist coverage of 0.90. Approximate 68 percent uncertainty due to sampling 
variability given for cases where estimated coverage probability is greater than or 

less than 1. ^ = 5. 



Us Bayesian FC RFC POE 



0.0 





.2 





.993± 





.002 


0.928± 


0.006 


0.928± 





.006 




1 






0.1 





.2 


0. 


.981± 





.003 


0.882± 


0.007 


0.888± 


0. 


.007 




1 






0.2 





.2 





.974± 


0. 


.004 


0.955± 


0.005 


0.955± 





.005 


0, 


,999± 


0, 


,001 


1.0 





.2 





.971± 





.004 


0.934± 


0.006 


0.934± 





.006 


0, 


,682± 


0, 


.010 


2.0 





.2 





.973± 





.004 


0.967± 


0.004 


0.967± 





.004 


0, 


,866± 





.008 


5.0 





.2 





.909± 





.006 


0.930± 


0.006 


0.930± 





.006 


0, 


,868± 


0, 


.008 


10.0 





.2 


0. 


.892± 


0. 


.007 


0.901± 


0.007 


0.901± 


0. 


.007 


0, 


912± 





,006 


20.0 





.2 





.892± 





.007 


0.894± 


0.007 


0.894± 





.007 


0, 


,907± 


0, 


.006 


0.0 


1 


.0 





.964± 





.004 


0.924± 


0.006 


0.924± 





.006 


0, 


,997± 





.001 


0.1 


1 


.0 





.956± 


0. 


.005 


0.911± 


0.006 


0.939± 





.005 


0, 


916± 


0, 


.006 


0.2 


1 


.0 





.969± 





.004 


0.929± 


0.006 


0.929± 


0. 


.006 


0, 


,872± 





,007 


1.0 


1 


.0 





.967± 





.004 


0.936± 


0.005 


0.937± 





,005 


0, 


,850± 





,008 


2.0 


1 


.0 





.968± 





.004 


0.912± 


0.006 


0.915± 





.006 


0, 


,902± 


0, 


.007 


5.0 


1 


.0 





,912± 





.006 


0.929± 


0.006 


0.929± 





.006 


0. 


,922± 





.006 


10.0 


1 


.0 





.893± 


0. 


.007 


0.909± 


0.006 


0.909± 





,006 


0, 


,909± 


0, 


,006 


20.0 


1 


.0 


0. 


,891± 


0. 


.007 


0.897± 


0.007 


0.902± 


0. 


.007 


0, 


901± 





,007 


0.0 


2 


.0 





,944± 





.005 


0.906± 


0.007 


0.908± 





.006 


0. 


996± 


0, 


.001 


0.1 


2 


.0 





.954± 





.005 


0.912± 


0.006 


0.938± 





.005 


0, 


,877± 





.007 


0.2 


2 


.0 





.944± 


0. 


.005 


0.902± 


0.007 


0.907± 





.006 


0, 


,884± 


0, 


.007 


1.0 


2 


.0 





,963± 





.004 


0.910± 


0.006 


0.922± 


0. 


.006 


0. 


,920± 





,006 


2.0 


2 


.0 





,966± 





.004 


0.921± 


0.006 


0.926± 





.006 


0. 


,902± 


0, 


.007 


5.0 


2 


.0 


0. 


,901± 





.007 


0.919± 


0.006 


0.920± 





.006 


0, 


,910± 


0, 


.006 


10.0 


2 


.0 





.902± 





.007 


0.909± 


0.006 


0.913± 





.006 


0, 


914± 





.006 


20.0 


2 


.0 





.897± 


0. 


.007 


0.902± 


0.007 


0.905± 





.007 


0. 


,908± 


0, 


.006 
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Table 4. Estimated coverage probabilities of uncertainty intervals with nominal 
frequentist coverage of 0.90. Approximate 68 percent uncertainty due to sampling 
variability given for cases where estimated coverage probability is greater than or 

less than 1. 2k = 25. 



Us Bayesian FC RFC POE 



0.0 





.2 





.982± 





.003 


0.973± 


0.004 


0.973± 





.004 




1 






0.1 





.2 


0. 


.974± 





.004 


0.967± 


0.004 


0.967± 


0. 


.004 




1 






0.2 





.2 





.953± 


0. 


.005 


0.937± 


0.005 


0.937± 





.005 




1 






1.0 





.2 





.969± 





.004 


0.954± 


0.005 


0.954± 





.005 


0, 


,698± 


0, 


.010 


2.0 





.2 





.974± 





.004 


0.979± 


0.003 


0.979± 





.003 


0. 


,884± 





.007 


5.0 





.2 





.925± 





.006 


0.928± 


0.006 


0.928± 





.006 


0, 


891± 


0, 


.007 


10.0 





.2 


0. 


.890± 


0. 


.007 


0.906± 


0.007 


0.906± 


0. 


.007 


0, 


,920± 





,006 


20.0 





.2 





.906± 


0. 


.007 


0.909± 


0.006 


0.909± 





.006 


0, 


920± 


0, 


.006 


0.0 


1 


.0 





.957± 





.005 


0.940± 


0.005 


0.940± 





.005 


0, 


,999± 





.001 


0.1 


1 


.0 





.958± 


0. 


.005 


0.931± 


0.006 


0.939± 





.005 


0, 


665± 


0, 


.011 


0.2 


1 


.0 





.960± 





.004 


0.931± 


0.006 


0.931± 


0. 


.006 


0, 


709± 





,010 


1.0 


1 


.0 





.961± 





.004 


0.946± 


0.005 


0.946± 





,005 


0, 


,875± 





,007 


2.0 


1 


.0 





.961± 





.004 


0.917± 


0.006 


0.917± 





.006 


0, 


916± 


0, 


.006 


5.0 


1 


.0 





,900± 





.007 


0.927± 


0.006 


0.927± 





.006 


0. 


,927± 





.006 


10.0 


1 


.0 





.910± 


0. 


.006 


0.930± 


0.006 


0.930± 





,006 


0. 


914± 


0, 


,006 


20.0 


1 


.0 





.907± 


0. 


.006 


0.914± 


0.006 


0.914± 





.006 


0, 


918± 





,006 


0.0 


2 


.0 





,948± 





.005 


0.929± 


0.006 


0.934± 





.006 


0. 


,998± 


0, 


.001 


0.1 


2 


.0 





,946± 





.005 


0.930± 


0.006 


0.933± 





.006 


0, 


,873± 





.007 


0.2 


2 


.0 





,949± 


0. 


.005 


0.928± 


0.006 


0.928± 





,006 


0, 


,889± 


0, 


.007 


1.0 


2 


.0 





.957± 


0. 


.005 


0.919± 


0.006 


0.924± 


0. 


.006 


0. 


909± 





,006 


2.0 


2 


.0 





.957± 


0. 


.005 


0.930± 


0.006 


0.931± 





.006 


0. 


,902± 


0, 


.007 


5.0 


2 


.0 





,881± 





.007 


0.913± 


0.006 


0.913± 


0. 


.006 


0, 


,896± 


0, 


.007 


10.0 


2 


.0 





.892± 





.007 


0.908± 


0.006 


0.909± 





.006 


0. 


,897± 





.007 


20.0 


2 


.0 





.900± 


0. 


.007 


0.909± 


0.006 


0.909± 





.006 


0, 


912± 


0, 


.006 
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Table 5. Estimated coverage probabilities of uncertainty intervals with nominal 
frequentist coverage of 0.95. Approximate 68 percent uncertainty due to sampling 
variability given for cases where estimated coverage probability is greater than or 

less than 1. ^ = 1. 



Us 


Hb 


Bayesian 




FC 


RFC 




POE 




0.0 





.2 


1 


0. 


,857± 


0.008 


0.857± 





.008 


1 






0.1 





.2 


1 


0. 


976± 


0.003 


0.976± 


0. 


.003 


1 






0.2 





.2 


1 


0. 


,948± 


0.005 


0.948± 





,005 


1 






1.0 





.2 


1 


0. 


974± 


0.004 


0.974± 





.004 


0.755± 


0, 


010 


2.0 





.2 


0.998± 0.001 


0. 


976± 


0.003 


0.977± 





.003 


0.892± 


0, 


,007 


5.0 





.2 


0.962± 0.004 


0. 


972± 


0.004 


0.974± 





.004 


0.953± 


0, 


005 


10.0 





.2 


0.951± 0.005 


0. 


949± 


0.005 


0.949± 


0. 


.005 


0.935± 


0, 


.006 


20.0 





.2 


0.949± 0.005 


0. 


953± 


0.005 


0.955± 





.005 


0.950± 


0, 


,005 


0.0 


1 


.0 


0.999± 0.001 


0. 


,748± 


0.010 


0.755± 





.010 


0.999± 


0, 


001 


0.1 


1 


.0 


0.998± 0.001 


0. 


,878± 


0.007 


0.886± 





.007 


0.995± 


0. 


,002 


0.2 


1 


.0 


0.997± 0.001 


0. 


,862± 


0.008 


0.870± 


0. 


.008 


0.998± 


0, 


.001 


1.0 


1 


.0 


0.998± 0.001 


0. 


945± 


0.005 


0.945± 





,005 


0.940± 


0, 


,005 


2.0 


1 


.0 


0.993± 0.002 


0. 


949± 


0.005 


0.956± 





.005 


0.946± 


0, 


005 


5.0 


1 


.0 


r\ c\r i >~7 i r\ r\r\ a 

0.967± 0.004 


0. 


951± 


0.005 


0.969± 





C\C\ A 

.004 


0.958± 


0, 


r\r\ a 

004 


10.0 


1 


.0 


0.958± 0.004 


0. 


959± 


0.004 


0.962± 





,004 


0.960± 


0, 


,004 


20.0 


1 


.0 


0.954± 0.005 


0. 


944± 


0.005 


0.955± 


0. 


.005 


0.950± 


0, 


,005 


0.0 


2 


.0 


0.993± 0.002 


0. 


,835± 


0.008 


0.864± 





.008 


0.994± 


0, 


,002 


0.1 


2 


.0 


0.988± 0.002 


0. 


869± 


0.008 


0.900± 





.007 


0.981± 


0, 


003 


0.2 


2 


.0 


0.990± 0.002 


0. 


856± 


0.008 


0.887± 





.007 


0.989± 


0, 


,002 


1.0 


2 


.0 


0.993± 0.002 


0. 


921± 


0.006 


0.928± 


0. 


.006 


0.965± 


0, 


,004 


2.0 


2 


.0 


0.991± 0.002 


0. 


935± 


0.006 


0.948± 





.005 


0.966± 


0, 


004 


5.0 


2 


.0 


0.967± 0.004 


0. 


937± 


0.005 


0.957± 





,005 


0.962± 


0, 


004 


10.0 


2 


.0 


0.944± 0.005 


0. 


943± 


0.005 


0.958± 





.004 


0.958± 


0, 


004 


20.0 


2 


.0 


0.952± 0.005 


0. 


945± 


0.005 


0.957± 





.005 


0.958± 


0, 


,005 
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Table 6. Estimated coverage probabilities of uncertainty intervals with nominal 
frequentist coverage of 0.95. Approximate 68 percent uncertainty due to sampling 
variability given for cases where estimated coverage probability is greater than or 

less than 1. ^ = 5. 



Us /is Bayesian FC RFC POE 



0.0 





.2 





.997± 





.001 


0. 


932± 


0. 


.006 


0. 


932± 





.006 


1 






0.1 





.2 





.997± 





.001 


0. 


972± 


0. 


.004 


0, 


,972± 





.004 


1 






0.2 





.2 





.993± 





.002 


0. 


974± 


0. 


.004 


0. 


,974± 


0, 


.004 


1 






1.0 





.2 





.990± 


0. 


.002 


0. 


971± 





,004 


0. 


971± 





.004 


0.684± 


0, 


,010 


2.0 





.2 





.983± 





.003 


0. 


976± 


0. 


.003 


0. 


,976± 





.003 


0.871± 


0, 


.007 


5.0 





.2 





.950± 





.005 


0. 


962± 





.004 


0. 


962± 





.004 


0.936± 





.005 


10.0 





.2 





.949± 





,005 


0. 


950± 


0. 


.005 


0. 


950± 





.005 


0.933± 


0, 


.006 


20.0 





.2 





.945± 


0. 


.005 


0. 


951± 


0. 


.005 


0. 


,955± 


0, 


.005 


0.950± 





,005 


0.0 


1 


.0 


0. 


,983± 





.003 


0. 


952± 


0. 


.005 


0, 


962± 





.004 


1 






0.1 


1 


.0 





.983± 





.003 


0. 


953± 





.005 


0. 


955± 





.005 


0.955± 





.005 


0.2 


1 


.0 





.986± 


0. 


.003 


0. 


953± 





,005 


0. 


,969± 





.004 


0.921± 


0, 


,006 


1.0 


1 


.0 





.984± 





.003 


0. 


967± 


0. 


.004 


0. 


,967± 


0, 


.004 


0.855± 





,008 


2.0 


1 


.0 





.982± 





.003 


0. 


968± 





,004 


0. 


,969± 





,004 


0.930± 





,006 


5.0 


1 


.0 





.961± 





.004 


0. 


966± 


0. 


.004 


0, 


966± 





.004 


0.945± 


0, 


.005 


10.0 


1 


.0 





.950± 





.005 


0. 


959± 


0. 


.004 


0. 


,959± 





.004 


0.955± 





.005 


20.0 


1 


.0 





,946± 


0. 


.005 


0. 


946± 





,005 


0. 


,950± 





,005 


0.941± 


0, 


,005 


0.0 


2 


.0 





.980± 


0. 


.003 


0. 


939± 





,005 


0. 


,957± 





.005 


0.999± 


0, 


.001 


0.1 


2 


.0 





.980± 





.003 


0. 


954± 


0. 


.005 


0, 


969± 





.004 


0.895± 


0, 


.007 


0.2 


2 


.0 





.978± 





.003 


0. 


943± 





.005 


0. 


,948± 





.005 


0.902± 





.007 


1.0 


2 


.0 





.977± 


0. 


.003 


0. 


951± 


0. 


,005 


0. 


,963± 





,004 


0.945± 


0, 


.005 


2.0 


2 


.0 





.985± 





.003 


0, 


959± 


0. 


,004 


0. 


,965± 


0, 


.004 


0.927± 





,006 


5.0 


2 


.0 





,960± 





.004 


0. 


957± 


0. 


.005 


0. 


,958± 





.005 


0.943± 


0, 


.005 


10.0 


2 


.0 





,945± 





.005 


0. 


959± 


0. 


.004 


0, 


,959± 





.004 


0.954± 


0, 


.005 


20.0 


2 


.0 





,944± 





.005 


0. 


947± 





.005 


0. 


,949± 





.005 


0.947± 





.005 
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Table 7. Estimated coverage probabilities of uncertainty intervals with nominal 
frequentist coverage of 0.95. Approximate 68 percent uncertainty due to sampling 
variability given for cases where estimated coverage probability is greater than or 

less than 1. 2k = 25. 



Us Bayesian FC RFC POE 



0.0 





.2 





.990± 





.002 


0.983± 


0.003 


0.983± 





.003 




1 






0.1 





.2 


0. 


,987± 





.003 


0.978± 


0.003 


0.978± 


0. 


.003 




1 






0.2 





.2 





.990± 


0. 


.002 


0.979± 


0.003 


0.979± 





.003 




1 






1.0 





.2 





,984± 





.003 


0.970± 


0.004 


0.970± 





.004 


0. 


,700± 


0, 


.010 


2.0 





.2 





.982± 





.003 


0.981± 


0.003 


0.981± 





.003 


0. 


,886± 





.007 


5.0 





.2 





.949± 





.005 


0.964± 


0.004 


0.964± 





.004 


0, 


,948± 


0, 


.005 


10.0 





.2 





.956± 


0. 


.005 


0.956± 


0.005 


0.956± 


0. 


.005 


0, 


934± 





,006 


20.0 





.2 





.948± 


0. 


.005 


0.956± 


0.005 


0.956± 





.005 


0, 


,958± 


0, 


.005 


0.0 


1 


.0 





.981± 





.003 


0.971± 


0.004 


0.972± 





.004 




1 






0.1 


1 


.0 





.978± 


0. 


.003 


0.970± 


0.004 


0.970± 





.004 


0, 


,687± 


0, 


.010 


0.2 


1 


.0 





.983± 





.003 


0.969± 


0.004 


0.973± 


0. 


.004 


0, 


,715± 





,010 


1.0 


1 


.0 





.984± 





.003 


0.967± 


0.004 


0.970± 





.004 


0, 


,876± 





,007 


2.0 


1 


.0 





.981± 





.003 


0.980± 


0.003 


0.980± 





.003 


0, 


949± 


0, 


.005 


5.0 


1 


.0 





,966± 





.004 


0.966± 


0.004 


0.966± 





.004 


0, 


,937± 





.005 


10.0 


1 


.0 





.955± 


0. 


.005 


0.972± 


0.004 


0.972± 





,004 


0, 


,965± 


0, 


,004 


20.0 


1 


.0 





.956± 


0. 


.005 


0.956± 


0.005 


0.956± 


0. 


.005 


0, 


,954± 





.005 


0.0 


2 


.0 





,978± 





.003 


0.964± 


0.004 


0.969± 





.004 




1 






0.1 


2 


.0 





.973± 





.004 


0.964± 


0.004 


0.967± 





.004 


0. 


,877± 





.007 


0.2 


2 


.0 





.979± 


0. 


.003 


0.964± 


0.004 


0.966± 





,004 


0, 


,894± 


0, 


.007 


1.0 


2 


.0 





.981± 





.003 


0.964± 


0.004 


0.967± 


0. 


.004 


0, 


946± 





,005 


2.0 


2 


.0 





.981± 





.003 


0.960± 


0.004 


0.961± 





.004 


0, 


911± 


0, 


.006 


5.0 


2 


.0 


0. 


,940± 





.005 


0.950± 


0.005 


0.949± 





.005 


0, 


929± 


0, 


.006 


10.0 


2 


.0 





,942± 





.005 


0.957± 


0.005 


0.957± 





.005 


0, 


941± 





.005 


20.0 


2 


.0 





,946± 


0. 


.005 


0.952± 


0.005 


0.952± 





.005 


0, 


,948± 


0, 


.005 
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Table 8. Estimated detection probabilities corresponding to uncertainty intervals with 
nominal frequentist coverage of 0.90. Approximate 68 percent uncertainty due to 
sampling variability given for cases where estimated coverage probability is greater 

than or less than I. — 1. 



Us Bayesian FC RFC POE 



0.0 





.2 









0. 


143± 


0.008 


0.143± 





.008 









0.1 





.2 









0. 


203± 


0.009 


0.202± 


0. 


.009 









0.2 





.2 


0.001± 


0. 


.001 


0. 


269± 


0.010 


0.268± 





,010 


0.001± 


0, 


,001 


1.0 





.2 


0.028± 





.004 


0. 


591± 


0.011 


0.580± 





.011 


0.027± 


0, 


.004 


2.0 





.2 


0.152± 





.008 


0. 


795± 


0.009 


0.759± 





,010 


0.144± 





,008 


5.0 





.2 


0.725± 





.010 


0. 


975± 


0.004 


0.951± 





.005 


0.695± 


0, 


.010 


10.0 





.2 


0.979± 


0. 


.003 




1 




0.998± 


0. 


.001 


0.971± 





,004 


20.0 





.2 


1 








1 




1 






1 






0.0 


1 


.0 


0.010± 





.002 


0. 


272± 


0.010 


0.252± 





.010 


0.009± 





.002 


0.1 


1 


.0 


0.010± 


0. 


.002 


0. 


293± 


0.010 


0.268± 





.010 


0.009± 


0, 


,002 


0.2 


1 


.0 


0.016± 





.003 


0. 


310± 


0.010 


0.280± 


0. 


.010 


0.015± 





,003 


1.0 


1 


.0 


0.070± 





.006 


0, 


461± 


0.011 


0.381± 





.011 


0.055± 





,005 


2.0 


1 


.0 


0.193± 





.009 


0. 


602± 


0.011 


0.495± 





.011 


0.157± 


0, 


.008 


5.0 


1 


.0 


0.673± 





.010 


0. 


887± 


0.007 


0.819± 





.009 


0.571± 





.011 


10.0 


1 


.0 


0.967± 


0. 


.004 


0. 


994± 


0.002 


0.985± 





.003 


0.936± 


0, 


,005 


20.0 


1 


.0 


1 








1 




1 






1 






0.0 


2 


.0 


0.036± 





,004 


0. 


214± 


0.009 


0.165± 





.008 


0.026± 


0, 


.004 


0.1 


2 


.0 


0.039± 





.004 


0. 


,238± 


0.010 


0.172± 





.008 


0.028± 





.004 


0.2 


2 


.0 


0.043± 


0. 


.005 


0. 


,253± 


0.010 


0.183± 





.009 


0.031± 


0, 


.004 


1.0 


2 


.0 


0.106± 





.007 


0. 


343± 


0.011 


0.249± 


0. 


.010 


0.073± 





,006 


2.0 


2 


.0 


0.228± 





.009 


0. 


493± 


0.011 


0.374± 





.011 


0.162± 





.008 


5.0 


2 


.0 


0.586± 





.011 


0. 


805± 


0.009 


0.702± 





,010 


0.461± 


0, 


.011 


10.0 


2 


.0 


0.918± 





.006 


0. 


,977± 


0.003 


0.949± 


0. 


.005 


0.850± 





.008 


20.0 


2 


.0 


0.999± 


0. 


.001 




1 




0.999± 





.001 


0.997± 


0, 


,001 
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Table 9. Estimated detection probabilities corresponding to uncertainty intervals with 
nominal frequentist coverage of 0.90. Approximate 68 percent uncertainty due to 
sampling variability given for cases where estimated coverage probability is greater 

than or less than I. — 5. 



Us Bayesian FC RFC POE 



0.0 





.2 





.007± 





.002 


0.072± 


0.006 


0.072± 





.006 











0.1 





.2 





.029± 





.004 


0.118± 


0.007 


0.118± 


0. 


.007 











0.2 





.2 





,045± 


0. 


.005 


0.148± 


0.008 


0.148± 





.008 


0. 


001± 


0. 


,001 


1.0 





.2 





.281± 





.010 


0.453± 


0.011 


0.453± 





.011 


0. 


022± 


0. 


.003 


2.0 





.2 





.549± 





.011 


0.694± 


0.010 


0.694± 


0. 


.010 


0. 


119± 





.007 


5.0 





.2 





.935± 





.006 


0.962± 


0.004 


0.962± 





.004 


0. 


654± 


0. 


.011 


10.0 





.2 


0. 


.998± 


0. 


.001 


0.999± 


0.001 


0.999± 


0. 


.001 


0. 


,977± 





,003 


20.0 





.2 




1 






1 




1 








1 






0.0 


1 


.0 





.036± 





.004 


0.076± 


0.006 


0.076± 





.006 


0. 


003± 





.001 


0.1 


1 


.0 





,048± 


0. 


.005 


0.089± 


0.006 


0.089± 





.006 


0. 


005± 


0. 


.001 


0.2 


1 


.0 





.055± 





.005 


0.103± 


0.007 


0.103± 


0. 


.007 


0. 


003± 





,001 


1.0 


1 


.0 





176± 





.009 


0.284± 


0.010 


0.283± 





.010 


0. 


023± 





,003 


2.0 


1 


.0 





.381± 





.011 


0.509± 


0.011 


0.509± 





.011 


0. 


114± 


0. 


.007 


5.0 


1 


.0 





.852± 





.008 


0.906± 


0.007 


0.906± 





.007 


0. 


569± 





.011 


10.0 


1 


.0 





,994± 





.002 


0.996± 


0.001 


0.995± 





.002 


0. 


960± 


0. 


,004 


20.0 


1 


.0 




1 






1 




1 








1 






0.0 


2 


.0 





.056± 





,005 


0.094± 


0.007 


0.092± 





.006 


0. 


004± 


0. 


.001 


0.1 


2 


.0 





,049± 





.005 


0.091± 


0.006 


0.088± 





.006 


0. 


003± 





.001 


0.2 


2 


.0 





,076± 


0. 


.006 


0.126± 


0.007 


0.120± 





.007 


0. 


007± 


0. 


,002 


1.0 


2 


.0 





153± 





.008 


0.226± 


0.009 


0.214± 


0. 


.009 


0. 


031± 





,004 


2.0 


2 


.0 





.318± 





.010 


0.412± 


0.011 


0.394± 





.011 


0. 


,094± 





.007 


5.0 


2 


.0 





.753± 





.010 


0.820± 


0.009 


0.803± 


0. 


.009 


0. 


,481± 


0. 


.011 


10.0 


2 


.0 


0. 


,984± 





.003 


0.988± 


0.002 


0.987± 


0. 


.003 


0. 


935± 





.006 


20.0 


2 


.0 




1 






1 




1 








1 
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Table 10. Estimated detection probabilities corresponding to uncertainty intervals 
with nominal frequentist coverage of 0.90. Approximate 68 percent uncertainty due to 
sampling variability given for cases where estimated coverage probability is greater 

than or less than 1. -^f = 25. 



Us Bayesian FC RFC POE 



0.0 





.2 





.018± 





.003 


0.027± 


0.004 


0.027± 





.004 











0.1 





.2 





.043± 





.005 


0.067± 


0.006 


0.067± 


0. 


.006 











0.2 





.2 





,079± 


0. 


.006 


0.099± 


0.007 


0.099± 





,007 


0. 


001± 


0, 


,001 


1.0 





.2 





.364± 





.011 


0.397± 


0.011 


0.397± 





.011 


0. 


,022± 


0, 


.003 


2.0 





.2 


0. 


.638± 





.011 


0.666± 


0.011 


0.666± 





.011 


0. 


136± 





.008 


5.0 





.2 





.968± 





.004 


0.970± 


0.004 


0.970± 





.004 


0, 


689± 


0, 


.010 


10.0 





.2 




1 






1 




1 






0, 


,983± 





,003 


20.0 





.2 




1 






1 




1 








1 






0.0 


1 


.0 





043± 





.005 


0.060± 


0.005 


0.060± 





.005 


0. 


001± 


0, 


.001 


0.1 


1 


.0 





.055± 





.005 


0.081± 


0.006 


0.081± 





.006 


0. 


,002± 





.001 


0.2 


1 


.0 





,076± 





.006 


0.098± 


0.007 


0.098± 





.007 


0, 


,004± 





.001 


1.0 


1 


.0 





.209± 





.009 


0.261± 


0.010 


0.261± 


0. 


.010 


0, 


,022± 





,003 


2.0 


1 


.0 





,417± 


0. 


.011 


0.493± 


0.011 


0.493± 





,011 


0, 


,097± 


0, 


,007 


5.0 


1 


.0 





.887± 





.007 


0.915± 


0.006 


0.915± 





.006 


0, 


,582± 


0, 


.011 


10.0 


1 


.0 





.997± 





.001 


0.997± 


0.001 


0.997± 





.001 


0, 


,972± 





.004 


20.0 


1 


.0 




1 






1 




1 








1 






0.0 


2 


.0 


0. 


.052± 


0. 


.005 


0.071± 


0.006 


0.066± 


0. 


.006 


0, 


002± 





,001 


0.1 


2 


.0 





,059± 


0. 


.005 


0.079± 


0.006 


0.075± 





.006 


0, 


006± 


0, 


.002 


0.2 


2 


.0 





.073± 





,006 


0.092± 


0.006 


0.089± 





.006 


0, 


,008± 


0, 


.002 


1.0 


2 


.0 





162± 





.008 


0.203± 


0.009 


0.197± 





.009 


0. 


,029± 





.004 


2.0 


2 


.0 





.347± 


0. 


.011 


0.404± 


0.011 


0.395± 





.011 


0, 


,098± 


0, 


.007 


5.0 


2 


.0 





.788± 





.009 


0.823± 


0.009 


0.822± 


0. 


.009 


0. 


,509± 





,011 


10.0 


2 


.0 





,989± 





.002 


0.991± 


0.002 


0.991± 





.002 


0, 


,933± 





.006 


20.0 


2 


.0 




1 






1 




1 








1 
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Table 11. Estimated detection probabilities corresponding to uncertainty intervals 
with nominal frequentist coverage of 0.95. Approximate 68 percent uncertainty due to 
sampling variability given for cases where estimated coverage probability is greater 

than or less than I. — 1. 



Us Bayesian FC RFC POE 



0.0 





.2 











0. 


143± 


0.008 


0.143± 





.008 











0.1 





.2 











0. 


202± 


0.009 


0.202± 


0. 


.009 











0.2 





.2 











0. 


268± 


0.010 


0.268± 





,010 











1.0 





.2 





,008± 





.002 


0. 


580± 


0.011 


0.575± 





.011 


0. 


008± 


0, 


.002 


2.0 





.2 


0. 


,060± 





.005 


0. 


759± 


0.010 


0.740± 





.010 


0, 


,059± 





.005 


5.0 





.2 





,534± 





.011 


0. 


951± 


0.005 


0.920± 





.006 


0, 


,515± 


0, 


.011 


10.0 





.2 





.949± 


0. 


.005 


0. 


998± 


0.001 


0.994± 





.002 


0, 


929± 





,006 


20.0 





.2 




1 








1 




1 








1 






0.0 


1 


.0 





001± 





.001 


0. 


252± 


0.010 


0.245± 





.010 


0. 


002± 





.001 


0.1 


1 


.0 





,003± 


0. 


.001 


0. 


268± 


0.010 


0.260± 





.010 


0, 


003± 


0, 


.001 


0.2 


1 


.0 





.004± 





.001 


0. 


280± 


0.010 


0.272± 


0. 


.010 


0, 


004± 





,001 


1.0 


1 


.0 





,017± 





.003 


0. 


381± 


0.011 


0.343± 





,011 


0. 


015± 





,003 


2.0 


1 


.0 





.082± 





.006 


0. 


495± 


0.011 


0.424± 





.011 


0. 


,072± 


0, 


.006 


5.0 


1 


.0 





.468± 





.011 


0. 


820± 


0.009 


0.723± 





.010 


0. 


379± 





.011 


10.0 


1 


.0 





,917± 





.006 


0. 


.985± 


0.003 


0.963± 





.004 


0, 


,844± 


0, 


,008 


20.0 


1 


.0 




1 








1 




1 






0, 


999± 





,001 


0.0 


2 


.0 





.007± 





.002 


0. 


165± 


0.008 


0.136± 





.008 


0, 


006± 


0, 


.002 


0.1 


2 


.0 





.013± 





.003 


0. 


172± 


0.008 


0.142± 





.008 


0. 


012± 





.002 


0.2 


2 


.0 





,011± 


0. 


.002 


0. 


183± 


0.009 


0.152± 





.008 


0. 


009± 


0, 


.002 


1.0 


2 


.0 





.033± 





.004 


0. 


249± 


0.010 


0.188± 


0. 


.009 


0, 


,025± 





,003 


2.0 


2 


.0 





100± 





.007 


0. 


.374± 


0.011 


0.279± 





.010 


0, 


066± 





.006 


5.0 


2 


.0 





.405± 





.011 


0. 


.702± 


0.010 


0.580± 





.011 


0, 


,285± 


0, 


.010 


10.0 


2 


.0 





.837± 





.008 


0. 


951± 


0.005 


0.903± 





.007 


0, 


,742± 





.010 


20.0 


2 


.0 





.997± 


0. 


.001 


0. 


999± 


0.001 


0.998± 





.001 


0. 


993± 


0, 


,002 
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Table 12. Estimated detection probabilities corresponding to uncertainty intervals 
with nominal frequentist coverage of 0.95. Approximate 68 percent uncertainty due to 
sampling variability given for cases where estimated coverage probability is greater 



than or less than I. — 5. 



Us /is Bayesian FC RFC POE 



0.0 





.2 





.003± 





.001 


0. 


068± 


0. 


.006 


0. 


,068± 





.006 









0.1 





.2 





.020± 





.003 


0. 


112± 


0. 


.007 


0. 


112± 





.007 









0.2 





.2 


0. 


.026± 





.004 


0. 


140± 


0. 


.008 


0. 


139± 


0, 


.008 









1.0 





.2 





196± 





.009 


0. 


412± 


0. 


,011 


0, 


410± 





,011 


0.005± 


0, 


,002 


2.0 





.2 





.441± 





.011 


0. 


639± 


0. 


.011 


0, 


,635± 





.011 


0.048± 


0, 


.005 


5.0 





.2 





.896± 





.007 


0. 


945± 


0. 


.005 


0, 


,943± 





.005 


0.480± 





.011 


10.0 





.2 





.997± 





,001 


0. 


999± 


0. 


.001 


0, 


998± 





.001 


0.947± 


0, 
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Table 13. Estimated detection probabilities corresponding to uncertainty intervals 
with nominal frequentist coverage of 0.95. Approximate 68 percent uncertainty due to 
sampling variability given for cases where estimated coverage probability is greater 

than or less than 1. -^f = 25. 



Us Bayesian FC RFC POE 



0.0 





.2 





.010± 





.002 


0.017± 


0.003 


0.017± 





.003 











0.1 





.2 


0. 


.024± 





.003 


0.042± 


0.004 


0.042± 


0. 


.004 











0.2 
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.005 
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0.006 
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Table 14. Root-mean-square deviation between observed and nominal frequentist 
coverage averaged over all values of fi s < 10. Nominal frequentist coverage is 0.90. 



% \i B Bayesian FC RFC POE 



1 


0. 


.2 





.084 


0. 


.052 


0. 


053 


0. 


.087 
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1 


.0 





,075 
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.101 


0, 


090 
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Table 15. Root-mean-square deviation between observed and nominal frequentist 
coverage averaged over all values of fi s < 10. Nominal frequentist coverage is 0.95. 



% Hb Bayesian FC RFC POE 
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Figure 1. Probability density functions for estimated background for case where 
Tbg/T — 25. For [i B = 0.2, 1, and 2, the standard deviations of the estimated 
background are 0.09, 0.20 and 0.28 respectively. The corresponding fractional standard 
deviations of the estimated background are 45 %, 20 %, and 14 % respectively. 
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Figure 2. Posterior probability density functions for the background parameter [Xb 
given the observed value nb g - 
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Figure 3. Estimated coverage probabilities for case where \ib 
nominal frcquentist coverage probability of 0.90. 
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Figure 4. Estimated detection probabilities for case where //b 
nominal frcquentist coverage probability of 0.90. 
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Figure 5. Estimated detection probabilities for case where there is no signal (us 
associated with intervals with nominal frequentist coverage probability of 0.90. 
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Figure 6. Estimated detection probabilities for case where there is no signal (/is = 0) 
associated with intervals with nominal frequentist coverage probability of 0.95. 
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Figure 7. Estimated coverage probabilities corresponding to intervals with target 
coverage of 0.90. In the plots, we show results for [lb = 0.2,1,2 and 7& g /T = 
1,2,5,10,25. 
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Figure 8. Estimated coverage probabilities corresponding to intervals with target 
coverage of 0.95. In the plots, we show results for [lb = 0.2,1,2 and 7& g /T = 
1,2,5,10,25. 
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Figure 9. Estimated detection probabilities corresponding to intervals with target 
coverage of 0.90. In the plots, we show results for [lb — 0.2,1,2 and Tb g /T = 
1,2,5,10,25. 
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Figure 10. Estimated detection probabilities corresponding to intervals with target 
coverage of 0.95. In the plots, we show results for /is = 0.2,1,2 and Tb g /T = 
1,2,5,10,25. 



